[BUG]: Added ServerAPI imports if API import fails #12
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
WIP: testing needed. |
|
Results from testing:
Clickhouse server migrationServer is running in 0.3.29 docker compose on chroma-migrate
((((((((( (((((####
((((((((((((((((((((((#########
((((((((((((((((((((((((###########
((((((((((((((((((((((((((############
(((((((((((((((((((((((((((#############
(((((((((((((((((((((((((((#############
(((((((((((((((((((((((((##############
((((((((((((((((((((((((##############
(((((((((((((((((((((#############
((((((((((((((((##############
((((((((( #########
Welcome to the Chroma Migration Tool
This tool will help you migrate your data from versions less than v0.4.0 to the latest version of Chroma.
Please visit https://docs.trychroma.com/migration for more information or join our discord channel #migrations at https://discord.gg/8g5FESbj for help
Loading existing collections...
Validating collection metadata...
Migrating existing collections...
Migrating existing embeddings...
0%| | 0/2 [00:00<?, ?it/s]
Traceback (most recent call last):
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/bin/chroma-migrate", line 8, in <module>
sys.exit(run_cli())
^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/chroma_migrate/cli.py", line 139, in run_cli
did_migration = migrate_from_clickhouse(api, clickhouse_host, clickhouse_port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/chroma_migrate/import_clickhouse.py", line 68, in migrate_from_clickhouse
metadata = json.loads(record[5])
^^^^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/.pyenv/versions/3.11.2/lib/python3.11/json/__init__.py", line 339, in loads
raise TypeError(f'the JSON object must be str, bytes or bytearray, '
TypeError: the JSON object must be str, bytes or bytearray, not NoneTypeLegacy 0.3.29 Serverchroma-migrate
((((((((( (((((####
((((((((((((((((((((((#########
((((((((((((((((((((((((###########
((((((((((((((((((((((((((############
(((((((((((((((((((((((((((#############
(((((((((((((((((((((((((((#############
(((((((((((((((((((((((((##############
((((((((((((((((((((((((##############
(((((((((((((((((((((#############
((((((((((((((((##############
((((((((( #########
Welcome to the Chroma Migration Tool
This tool will help you migrate your data from versions less than v0.4.0 to the latest version of Chroma.
Please visit https://docs.trychroma.com/migration for more information or join our discord channel #migrations at https://discord.gg/8g5FESbj for help
Traceback (most recent call last):
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/fastapi.py", line 628, in raise_chroma_error
resp.raise_for_status()
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 404 Client Error: Not Found for url: http://localhost:8000/api/v1/tenants/default_tenant
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/client.py", line 427, in _validate_tenant_database
self._admin_client.get_tenant(name=tenant)
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/client.py", line 472, in get_tenant
return self._server.get_tenant(name=name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/telemetry/opentelemetry/__init__.py", line 127, in wrapper
return f(*args, **kwargs)
^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/fastapi.py", line 198, in get_tenant
raise_chroma_error(resp)
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/fastapi.py", line 630, in raise_chroma_error
raise (Exception(resp.text))
Exception: {"detail":"Not Found"}
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/bin/chroma-migrate", line 8, in <module>
sys.exit(run_cli())
^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/chroma_migrate/cli.py", line 147, in run_cli
from_chroma = chromadb.HttpClient(host=chroma_source_host, port=chroma_source_port)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/__init__.py", line 181, in HttpClient
return ClientCreator(tenant=tenant, database=database, settings=settings)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/client.py", line 143, in __init__
self._validate_tenant_database(tenant=tenant, database=database)
File "/Users/tazarov/experiments/chroma-experiments/oss/1452-migration-issue-chroma-0418/.venv/lib/python3.11/site-packages/chromadb/api/client.py", line 433, in _validate_tenant_database
raise ValueError(
ValueError: Could not connect to tenant default_tenant. Are you sure it exists?Due to the validation of tenant and DB direct migration from 0.3.x to 0.4.15+ (or from the first time we introduced multi-tenancy) ConclusionWe should update the instructions that the upgrade path should be 0.3.x->0.4.14 and then 0.4.14<= to 0.4.15+. @HammadB wdyt? |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Refs: chroma-core/chroma#1452